Technology of capturing real space using a capture device mounted on a moving object, and representing the captured real space as virtual space using a computer based on the captured image data (for example, ‘Building Image-Based Cybercities by Using Vehicle-Mounted Cameras’ by Endoh, Katayama, Tamura, Hirose, Watanabe, and Tanigawa (published by Shingaku Society, PA-3–4, pp. 276–277, in 1997), ‘Building Image-Based Cybercities by Using Vehicle-Mounted Cameras (2)—Generation of Wide-Range Virtual Environment by Using Photorealistic Images—’ by Hirose, Watanabe, Tanigawa, Endoh, Katayama, and Tamura (published in Proceedings of the Virtual Reality Society of Japan Second Annual Conference, pp. 67–70, in 1997), etc.
A method for representing real space captured as virtual space based on the captured image data captured by the capture device mounted on a moving object can be a method of reconstructing a geometric model of real space based on captured image data, and representing it in the conventional CG technology. However, the technology is limited in correctness, precision, reality, etc. of a model. On the other hand, the image-based rendering (IBR) technology of representing virtual space using a captured image without using a model has recently attracted attention. The IBR technology produces an image from any viewpoint based on a plurality of captured images. Since the IBR technology uses captured images, realistic virtual space can be represented.
To produce walk-through system of virtual space using the above mentioned IBR technology, it is necessary to produce and present an image depending on the position in the virtual space of a user. In this type of system, each frame of captured image data is associated and stored with the position in the virtual space, the corresponding frame is obtained based on the position and the view direction of the user in the virtual space, thereby reproducing the frames.
As a common method of obtaining the position data in the real space, the position measurement system using an artificial satellite represented by the GPS (Global Positioning System) which is also used for a car navigation system, etc. A method of associating the position data obtained in the GPS with the captured image data can use a time code (Japanese Patent Application laid-open No. 11-168754). In this method, each frame of captured image data can be associated with position data by associating the time data contained in the position data with the time code added to each frame of captured image data.
In the walk-through presentation in the above mentioned virtual space, a user can see a desired direction from his or her viewpoint. Therefore, an image at each viewpoint is stored as a panoramic image covering a range wider than an image to be reproduced, a partial image to be reproduced is cut out and displayed from the panoramic image based on the position and the direction of the viewpoint of the user in the virtual space.
However, an image actually displayed in the walk-through system is a part of a panoramic image. Although the other portions are not used, it is necessary to read data or perform a decoding process on the entire panoramic image, thereby performing an inefficient process. That is, to display an image, it is necessary to read the entire panoramic image depending on the position of a user, decode the entire panoramic image when it is compressed, cut out a partial image from the obtained image data depending on the direction of the view of the user, and then display the image. Although the reading and decoding processes are performed on the entire panoramic image, only a part of it is practically used for display, and other portions are wastefully processed.
Especially when a full round image or a full sky image is used as a panoramic image, most of the processes of reading and decoding the panoramic image at each viewpoint position are not required and wasteful when the image is actually displayed. That is, the computer is excessively loaded, thereby disabling the walk-through system in reproducing and displaying images in real time.